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- The MAILING DATE of this communication appears on the cover sheet with the correspondence address - 
Period for Reply 

A SHORTENED STATUTORY PERIOD FOR REPLY IS SET TO EXPIRE 3 MONTH(S) FROM 
THE MAILING DATE OF THIS COMMUNICATION. 

- Extensions of time may be available under the provisions of 37 CFR 1 .136(a). In no event, however, may a reply be timely filed 
after SIX (6) MONTHS from the mailing date of this communication. 

- If the period for reply specified above is less than thirty (30) days, a reply within the statutory minimum of thirty (30) days will be considered timely. 

- If NO period for reply is specified above, the maximum statutory period will apply and will expire SIX (6) MONTHS from the mailing date of this communication. 

- Failure to reply within the set or extended period for reply will, by statute, cause the application to become ABANDONED (35 U.S.C. § 133). 

- Any reply received by the Office later than three months after the mailing date of this communication, even if timely filed, may reduce any 
earned patent term adjustment. See 37 CFR 1.704(b). 

Status 

1 )^ Responsive to communication(s) filed on 23 October 2003 . 
2a)D This action is FINAL. 2b)El This action is non-final. 

3) D Since this application is in condition for allowance except for formal matters, prosecution as to the merits is 

closed in accordance with the practice under Ex parte Quayle, 1935 CD. 1 1 , 453 O.G. 21 3. 

Disposition of Claims 

4) ^] Claim(s) 1-24.26,28-30.32-38 and 40-44 is/are pending in the application. 

4a) Of the above claim(s) is/are withdrawn from consideration. 

5) D Claim(s) is/are allowed. 

6) |EI Claim(s) 1-24. 26. 28-30. 32-38.40-44 is/are rejected. 

7) D Claim(s) is/are objected to. 

8) D Claim(s) are subject to restriction and/or election requirement. 

Application Papers 

9) D The specification is objected to by the Examiner. 

10) D The drawing(s) filed on is/are: a)D accepted or b)D objected to by the Examiner. 

Applicant may not request that any objection to the drawing(s) be held in abeyance. See 37 CFR 1 .85(a). 
Replacement drawing sheet(s) including the correction is required if the drawing(s) is objected to. See 37 CFR 1.121(d). 

1 1) D The oath or declaration is objected to by the Examiner. Note the attached Office Action or form PTO-152. 
Priority under 35 U.S.C. §§119 and 120 

12) D Acknowledgment is made of a claim for foreign priority under 35 U.S.C. § 1 19(a)-(d) or (f). 
a)d All b)D Some*c)D None of: 

Certified copies of the priority documents have been received. 
Certified copies of the priority documents have been received in Application No. 



3-D 



Copies of the certified copies of the priority documents have been received in this National Stage 
application from the International Bureau (PCT Rule 17.2(a)). 
* See the attached detailed Office action for a list of the certified copies not received. 

13) D Acknowledgment is made of a claim for domestic priority under 35 U.S.C. § 1 19(e) (to a provisional application) 

since a specific reference was included in the first sentence of the specification or in an Application Data Sheet. 
37 CFR 1.78. 

a) □ The translation of the foreign language provisional application has been received. 

14) D Acknowledgment is made of a claim for domestic priority under 35 U.S.C. §§ 120 and/or 121 since a specific 

reference was included in the first sentence of the specification or in an Application Data Sheet. 37 CFR 1 .78. 



Attachment(s) 

1) □ Notice of References Cited (PTO-892) 

2) □ Notice of Draftsperson's Patent Drawing Review (PTO-948) 

3) □ Information Disclosure Statement(s) (PTO-1449) Paper No(s). 



4) □ Interview Summary (PTO-413) Paper No(s). 

5) □ Notice of Informal Patent Application (PTO-152) 

6) □ Other: 



U.S. Patent and Trademark Office 
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DETAILED ACTION 



Response to Amendment 



1 . In response to the Office Action mailed December 4, 2002 applicants have submitted an 
Amendment, filed October 23, 2003. 

Response to Arguments 

2. Applicants' arguments have been fully considered but they are not persuasive, for the 
following reasons: 

3 . Applicants argue a difference between Rtischev's reading quality score and their 
intelligibility estimate, the latter said to measure "the degree to which others can understand a 
person's speech" (Amendment, p. 4). They assert that "a teacher's evaluation of a speaker's 
intelligibility is a subjective evaluation" while "comparing a transcript of what a listener hears the 
speaker say with the items that were spoken... provides an objective evaluation of a speaker's 
intelligibility" (p. 6). 

However, Rtischev et al. actually teach using a speech recognizer to determine a reader's 
speech quality, which measure they state is "preferably based" on "reading errors" determined by 
a "context-sensitive speech recognizer" (Abstract). By applicants' own argument, the resultant 
speech-to-text transcription by the recognizer is then also a measure of the speaker's 
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intelligibility. Moreover, a speech recognizer is even more "objective" than a listener who 
makes a transcription, because, unlike the recognizer, the listener would not necessarily produce 
the identical transcript when listening to the identical utterance on different days (as can be 
readily proved by having the same person transcribe the same, reasonably long, recorded 
speech). 

True, "A speaker may obtain a high reading quality score" from a speech recognizer (or a 
teacher) "even though a listener may have difficulty understanding the speaker". But, the reverse 
inconsistency can occur with a speech recognizer trained to different speakers versus a listener 
familiar with the speaker's accent, so that this has nothing to do with any supposed difference 
between speech quality and intelligibility. 

Despite applicants' argument that "The ability to read and the ability to be understood by 
others are two different abilities that require different testing methods", the examiner still exists 
that Rtischev et al. and applicants are inescapably measuring both speech quality and 
intelligibility, for they go together, but their evaluations can vary with the background of the 
listener or the training procedure for the speech recognizer, in both cases depending on their 
respective degree of "tolerating strong foreign accents from a non-native user" (all quotes from 
Amendment, p. 4), which can vary. 

Furthermore, Rtischev et al. also teaches the FSM which rates the quality of an input, and 
hence, an intelligibility (see newly presented recitations below; and also col. 6 lines 12-65). 

As per the repeated arguments presented on pages 13-15, please see comments above, 
and the further defined rejection presented below. 
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4. As for the newly-added limitations to claims 1 and 24 of "preparing a transcription of 
what was heard" or "creating a transcription of what the listener hears", respectively, they are 
broad enough not to require the listener to be transcribing anything, for a speech recognizer 
located near a listener (or where a potential listener would be located) and doing speech-to-text 
transcription reads on these limitations. As indicated above, such a speech recognizer is taught 
by Rtischev et al. Also, their "script", which is being read for reading evaluation, also represents 
"what was heard" or "what the listener hears". 

5. Therefore, the amended pending claims stay rejected over Rtischev et al, and their 
rejection is repeated, mutatis mutandis for claim amendments and cancellations, below. 

6. As for the argument that co-pending application 09/3 1 1 ,6 1 7 does not involve 
obviousness-type double-patenting because "Townshend does not suggest or claim a method or 
apparatus for measuring intelligibility of a speaker using a transcription of what a listener hears", 
the examiner disagrees for reasons given above, indicating that "what a listener hears" is the 
same as what a speech recognizer determines. Thus, the provisional double-patenting rejection 
stands, and is repeated, mutatis mutandis, below. 

Claim Rejections - 35 USC 

7. The text of those sections of Title 35, U.S. Code not included in this action can be found 
in the previous Office action. 
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Claim Rejections - 35 USC §102 
8. Claims 1, 2, 5, 7, 8, 36, and 37 are rejected under 35 U.S.C. 102(b) as being anticipated 
by Dimitry Rtischev et al (U.S. Patent 5,634,086, issued May 27, 1997). 

As per claims 1, 2, 5, 7, 36, and 37, Rtischev et a/.teach: 

• a means for hearing at least one person who is repeating items (spoken-language 
instruction apparatus employing speech recognition with user reading words from a 
written script from an inherent database, Abstract; user and microphone, or user and 
telephone, Fig. 1, elements 12 and 16, or elements 12 and 14, respectively); and 

• means for preparing a transcription of what was heard by the means for hearing (as input 
speech being transcribed - col. 5 lines 5-27) 

• means for comparing the items with a transcription of what was heard and thus measuring 
intelligibility from the comparison (speech recognizer using nonlinear HMM speech 
models, Fig. 3, element 112; preselected script, element 114; score set, element 120; 
reading errors, col. 3, lines 43 and 47). 

• A measurement unit operable to determine an intelligibility score of the speaker by 
comparing the items and a transcription of what the listener hears when the speaker 
repeats the items (as FSM comparing the resubmitted sentence after determining the 
previous result wasn't satisfactory (col. 6 lines 54-67) 

As per claim 8, the "reading errors" (col. 3, lines 43 and 47) of Rtischev et al inherently 
include at least word substitutions, for an error in reading a word could cause the ASR to 
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interpret it as a different existing word (e.g. a Japanese reader using Rtischev et a/.'s apparatus to 
learn English might pronounce "frame" as "flame", which would cause the ASR to recognize the 
spoken word as the latter). 



Claim Rejections - 35 USC §103 

9. Claims 3, 4, 6, 8, 1 1-18, 24, 26, 28-30, 32, 33, 42, and 44 are rejected under 35 U.S.C. 
103(a) as being unpatentable over Rtischev et aL as applied to claims 1, 2, 5, 7, 8, 36, and 37, 
above. 

As per claims 3, 4, 8, 1 1-13, 15-18, 24, 26, 28-30, 32, 33, 42, and 44, Rtischev et al.do 
not teach a listener or a plurality of people capable of listening to the speaker. However, the 
examiner takes Official Notice that it is centuries old and notoriously well known to have 
teachers listen to speakers (students) so as to evaluate the intelligibility of their speech in terms 
of reading errors. Therefore, it would have been obvious for an artisan at the time of invention to 
have also people listening, at least during the training mode for the Automatic Speech 
Recognizer (cf. Fig. 3, elements 102 and 104), to make sure the ASR does not make 
unreasonable "recognitions" or rejections. (Claim 28 has been reinterpreted as depending on 
claim 24.) Furthermore, Rtischev et aL teaches the FSM listing and responding from the speaker 
(col. 6 lines 54-67), with further repetitions from the user. 

The rest of the limitations were discussed in connection with the rejection of claims 
1, 2, 5, 7, 8, 36, and 37, above. 
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As per claim 14, Rtischev et al do not teach selecting listeners based on certain 
background characteristics. However, it would have been obvious for an artisan at the time of 
invention to select listeners that have extensive background speaking knowledge of the language 
being learned because they would be best able to determine the intelligibility of someone trying 
to speak the language. 

10. Claims 9, 10, 19, 20-23, 25, 34, 35, 38-41, and 43 are rejected under 35 U.S.C. 103(a) as 
being unpatentable over Rtischev et al. as applied to claims 1, 2, 5, 7, 8, 36, and 37, above, in 
view of Charles Lewis et al (U.S. Patent 5,059,127, issued October 22, 1991). 

As per claims 9, 10, 19, 20-23, 25, 34, 35, 41, and 43, while Rtischev et al teach 
evaluating an error count intelligibility score (reading errors, col. 3, lines 43 and 47), they do not 
evaluate difficulty of the items and ability of a listener, nor doing this using Item Response 
Theory. However, Lewis et al. do (col. 1, line 63 through col. 2, lines 1, 16-26, and 42-43, with 
Figure 1A). 

It would have been obvious for an artisan at the time of invention to do this because 
Lewis et al. teach that IRT "allows creation of a test in which different individuals receive 
different questions, yet can be scored on a common scale" as well as "permits determination in 
advance of test administration of the level of ability and the accuracy with which ability has been 
measured" (col. 2, lines 31-36). 

The rest of the limitations were discussed in connection with the rejection of claims 
1, 2, 5, 7, 8, 36, and 37, above. 
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As per claim 38, Rtischev et al. do not teach a database containing data from previous 
evaluations. However, Lewis et al. teach retaining data from previous "testlets" and 
"sequentially administering testlets ... until a pass/fail decision can be made" (Abstract), thus 
suggesting retaining results of previous intelligibility evaluations (testlets) for later continued 
evaluation. It would have been obvious for an artisan at the time of invention to do this, to avoid 
having to administer all the testlets in a single sitting. 

As per claim 39, Rtischev et al. teach evaluations using data selected from at least 
speaker responses and items (col. 3, lines 11-17). 

As per claim 40, Rtischev et al. suggest use of nonlinear artificial neural net models for 
speech recognition (see reference to Kim et al. under "Other Publications" on the front page, top 
of second column). 
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Double Patenting 

11. The nonstatutory double patenting rejection is based on a judicially created doctrine 
grounded in public policy (a policy reflected in the statute) so as to prevent the unjustified or 
improper timewise extension of the "right to exclude" granted by a patent and to prevent possible 
harassment by multiple assignees. See In re Goodman, 1 1 F.3d 1046, 29 USPQ2d 2010 (Fed. 
Cir. 1993); In re Longi, 759 F.2d 887, 225 USPQ 645 (Fed. Cir. 1985); In re Van Ornum, 686 
F.2d 937, 214 USPQ 761 (CCPA 1982); In re Vogel, 422 F.2d 438, 164 USPQ 619 

(CCPA 1970);and, In re Thorington, 418 F.2d 528, 163 USPQ 644 (CCPA 1969). 

A timely filed terminal disclaimer in compliance with 37 CFR 1.321(c) may be used to 

overcome an actual or provisional rejection based on a nonstatutory double patenting ground 

provided the conflicting application or patent is shown to be commonly owned with this 

application. See 37 CFR 1.130(b). 

Effective January 1, 1994, a registered attorney or agent of record may sign a terminal 

disclaimer. A terminal disclaimer signed by the assignee must fully comply with 37 CFR 

3.73(b). 

12. Claims 1-24, 26, 28-30, and 32-44 are provisionally rejected under the judicially created 
doctrine of obviousness-type double patenting as being unpatentable over claims 1-3, 5-9, and 
11-15 of copending Application No. 09/311,617. Although the conflicting claims are not 
identical, they are not patentably distinct from each other because they claim obvious variations 
of the claims in the instant application. The operation of the speech recognizer constitutes 
preparing a transcript. 
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This is a provisional obviousness-type double patenting rejection because the conflicting 
claims have not in fact been patented. 



Conclusion 



14. Any response to this action should be mailed to: 

Commissioner of Patents and Trademarks 
Washington, D.C. 2023 1 
or faxed to: 
(703) 872 9314, 

(for informal or draft communications, please label "PROPOSED" or "DRAFT") 
Hand-delivered responses should be brought to Crystal Park II, 2121 Crystal 
Drive, Arlington. VA., Sixth Floor (Receptionist). 

Any inquiry concerning this communication or earlier communications from the examiner 
should be directed to Michael Opsasnick, telephone number (703)305-4089, who is available 
Tuesday-Thursday, 9AM-4PM. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Ms. Doris To, can be reached at (703)305-4827. The facsimile phone number for 
this group is (703)872-9314. 

Any inquiry of a general nature or relating to the status of this application or proceeding 
should be directed to the Group 2600 receptionist whose telephone number is (703) 305-4750, 
the 2600 Customer Service telephone number is (703) 306-0377. 



mno 

1/6/2004 




DORIS H. TO [ (^^ 
SUPERVISORY PATENT EXAMINER 
TECHNOLOGY CENTER 2800 



